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1, INTRODUCTION 

Optimization is a procedure of adjusting system characteristics to make it works effectively under 
some constraints or conditions. Optimization algorithms have been utilized in the various fields to solve the 
real-world optimization problem, and these algorithms have been designed to satisfy the requirement of an 
applied system [1]. For each optimization problem, there are set of possible solutions called solution scope 
[1]. The achievable solution founded by the algorithm may consider as a global optimum if it is the only 
better solution among other feasible solutions. Computing global optimal for the high dimensional 
optimization problem is complex [2], and considers being one of the most challenges facing the optimization 
algorithms. Since optimization problems appear in diverse fields including engineering, manufacture and the 
economic system, there is a necessary need for an efficient algorithm that could solve the high dimensional 
problem successfully [3]. 

Two main types of global optimization methods are founded in literature: deterministic and 
probabilistic [4]. Deterministic algorithms deal with the problem by making a clear assumption of it and 
exploring the search space in a suitable way to achieve a fixed solution within an acceptable amount of time 
[4]. These algorithms could not use in such problems that have large search area such as high dimensional 
optimization problem. In addition, due to time constraint, these methods have a less capability to explore the 
large search space [5]. However, probabilistic algorithms such as metaheuristic algorithms have shown 
advanced in the high dimensional optimization [6]. Metaheuristic algorithms are taken its essences from 
nature. Two characteristics have formed these algorithms: exploration and exploitation [6, 7]. Exploration 1s 
seeking the search space, while exploiting is utilizing the visited area to examine that the global solution 
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within the area [7]. Consequently, metaheuristic algorithms have a chance to achieve the global optimum 
solution [8, 9]. Many fields including computer science, Artificial Intelligence, machine learning, and data 
mining have used the metaheuristic algorithms in their optimization procedure [8]. Although the 
metaheuristic algorithms have the same concept of taking their idea from nature, these algorithms come with 
different search mechanisms [9], and various inspired sources [7]. For instance, algorithms such as genetic 
algorithms (GA), and differential evolution (DE) are bio-inspired based algorithm while particle swarm 
optimization (PSO), Ant Colony Optimization (ACO), and cuckoo search (CS) are swarm intelligence-based 
algorithm [10]. In this study, we have chosen swarm intelligence algorithms to deal with high dimensional 
optimization problem. There are more than one hundred heuristic algorithms and many of them could solve 
different type of optimization problems effectively [5]. Within the huge number of algorithms selecting one 
algorithm among others to apply it in a specific domain to solve the high dimensional problem is not an 
easily accomplished task. For that, the paper aims to determine the algorithm that could resolve the high 
dimensional problem properly. This article is focusing on analyzing, and comparing in details particle swarm 
optimization (PSO) and cuckoo search (CS) in respect of solution accuracy and runtime performance on 
standard benchmark functions. The organization of this paper is shown as follow: Section II provides the 
optimization problem description. Section III shows a review of (PSO), and (CS). The standard function 
using in experiments are in section IV, while the results and conclusion are presented in section V and VI 
respectively. 


2. HIGH DIMENSIONAL OPTIMIZATION PROBLEM 

High dimensional optimization has two main issues one is that within the increase of dimensions 
number, the number of possible solutions is grown extensively [1, 3]. And the other is that the search space 
extended exponentially [1, 3]. These two issues make the algorithm face a difficulty to achieve an optimal 
solution at the appropriate time [11]. Although optimization methods have been utilized in the various large- 
scale space problems including electronic systems designing, enormous resources scheduling, an effective 
solution for problems involving high dimensions is highly required [11]. With the rapid evolution and 
increases of data among various fields, the need to test an existing algorithm to find the suitable method that 
cope the high dimensional optimization problems is crucial. 


2.1 Problem Formulation 

Many crucial fields including engineering, medicine, and economics rely on optimization 
mechanism to achieve their requirements. The optimization of the problems fields could be represented using 
mathematical functions to be solved by computational methods [12]. Optimization problem expressed by: 

1) The cost function objective function or represents the goals of optimization either minimize 
or maximize. 


f:XaY (1) 


Where Y should belong to the real number Yok , X represented the parameters or dimensions of 
the problem, and R is represented the search scope 


2) The dimensions or variables of the problem (% Xp 2049 %q) 

3) The constraints, which determine the boundary dimensions of a particular problem. 

In this study, the cost function is indicated by the fitness or quality of variables for a 
minimization problem. 


Minimize [0%) 1 = X29 Xn) (2) 


‘ is considered as a global solution to a given problem if the solution is better than any 
other solutions. 


3. INTELLIGENCE METHODS 

Swarm means a group of birds, ants or bees live in colonies [13, 14] in which the parts of the group 
communicate for varies tasks such as building a new nest or searching for food. Swarm intelligence 
algorithms are widely used for optimization problem among other algorithms; for instance, particle swarm 
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optimization and cuckoo search are applied in science and engineering, and these algorithms have the ability 
to tackle the varies types of optimization problems [6]. 


3.1. Particle Swarm Optimization 

Eberhart and Kennedy in 1995 proposed a swarm based intelligence algorithm named Particle 
swarm optimization (PSO) [15]. Particle swarm optimization functionality relay on the particles. Two 
characteristics: position and velocity belong to every single particle. The particles have the tow best position: 
personal (Pbest) and global (Gbest) respect to the whole group making the particle learn from its experience 
as well as from the whole group for searching an optimized solution. These poisons are updating based on the 
fitness value comparison between the current and the new position. Until the swarm finds the desired solution 
this process has been replicated many times. Three vectors are using for identifying particle in the search 
scope: position X,(t), velocity, V;(t) , and personal best position P,,., .In addition, its movement determined 


by P,.,, and G,,,,. PSO velocity and position formula are presented in equation 3,4 receptively. 


V.(t+)=V,(t)+c, *r *(P, 


est ~ & AE) Cy * I * (Gye - X;() (3) 


est 


X (t+ )=X,O+V,¢+) (4) 


Where, 7,,/, random numbers with values between (0,1) while C,, and C, are learning factors. 
PSO utilizes these factors to control the updating process of particle velocity and position. These two 
parameters are used to control the velocity and position of the particle. 

Particle swarm optimization has appealed many researchers over other algorithms. The algorithm is 
simple to implement and has less number of parameters [16]. On the other hand, it has some drawbacks that 
preventing the algorithm from effectively working on some optimization problem. The researchers solved 
this problem by some modification on the basic version. One of this modification is proposed by Shi and 
Eberhart in 1998, they introduced the Inertia Weight equals to 1 to control exploration and exploitation in 
search space [17], and the velocity equation is altered to (5). 


Vi(t+Daw* Vi(t)+c, *7, *(P,., -X(t) +6, *% *(G,,, -X (0) (5) 


est 


Because of the importance of the inertia weight on the PSO performance, it has been well studied in 
the literature [18]. In this study, the inertia weight is equal to w=0.7298 [19]. The Pseudocode of particle 
swarm optimization is as follows: 


For individual particle( * ) 
Initialize particle velocity 
Initialize particle position 
End 

Do 

For 1 =1 to population size 
Evaluate the fitness value 


If the current fitness value is better than the particle best value ( Es yea) 


assign current value to particle best value ( ie Bee 


end 
For each particle 


Find particle with the best fitness among all particles as ( G, Ben 


Update particle velocity according to equation (3) 
Update particle position according to equation (4) 
End 
While maximum iteration is not reached 


3.2. Cuckoo Search 

New based swarm intelligence algorithm named Cuckoo search(CS) has been announced in 2009 
[20]. cuckoo search takes its concepts from the cuckoo bird which depend on other birds to brood its eggs. 
Cuckoo search has presented advantage performance over many optimization problems; additionally, a study 
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has mentioned that this algorithm has an ability to find the global optimal [21]. CS has only two parameters 
to control its progress. That means it doesn’t need to regulate the parameter values for specific problems. For 
that, CS seems to be more generic for variation number of optimization problems [20]. Cuckoo search 
follows these steps mimicking the cuckoo birds. 

First, cuckoo birds select a random nest to put its eggs on it. Second, the nest with good merits will 
be transferred to the next production. Finally, the host birds have two choices either throw away the eggs or 
leave the nest to create a new one with the probability Pa © (0, 1). The algorithm implementation is 
performed based on the probability of using the new cuckoo solution [instead of the bad old solution.Lév y 
flight is invoked whenever there is new creating of the solution as shown in equation (6) and its step obtained 
by Lévy [20]. 


X (t+) =X,(t) +a @Lévy(A) (6) 


Where a>0 and set to | in cuckoo search and A is a parameter which determines the number of 
appearance throughout unit interval [21]. 


Based on steps above the CS procedure can summaries in the pseudocode: 
Begin 

Initialize population values of the host nest 

Do 

Generate new cuckoo solution using Levy flight and evaluate it 
If the cuckoo eggs better than the host eggs 

Replace the host eggs by cuckoo 

End if 

By detection fraction of (pa) as worst nest build new nest 

Save the best solutions and grade them 

Select the current best solution 

While maximum iteration is not reached 


4. RESEARCH METHOD 

Standard benchmark functions have been presented to validate the new algorithm as well as a 
comparison between several algorithms [22, 23]. These functions have a various category such as unimodal, 
multimodal, hybrid and comparison function. Three standard test functions are considered in this study 
including Ackley function, Rosenbrock function, and Rastrigin function. 

Ackley’s function, has many local minima making the algorithm faces difficulty to escape them. It 
is a multimodal function [22]. The two-dimension form is presented in Figure 1 and the evaluated formula in 
equation (7). 


f(x) = 20+ exp(1) — 20exp(—0.2, et) — exp(— >. cos(2IT x, yy @) 


Ackley’s function, has global minimum f (x )=Oat x =(0,0,...,0) with this constraint x,€[-5,5] for 
i=1,2, 3,..,D 


Ackley function 


f(x1,x2) 








Figure 1. Ackley Function of Two Dimensions 
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The Rosenbrock Function is a unimodal function which algorithm could easily find the range of 


global minimum; however, reaching the optimal solution is difficult. 


f(x) = Y[00(%,.-37)? +07 =] ®) 


Its global minimum f (x ) =(Qatx = (1,1,...,1) with this constraint x,€[—5,5] where for=1,2, 3,...D 


Rosenbrock function 





Figure 2. Rosenbrock Function fo of Two Dimensions 


Rastrigin’s function, which has many local minima and considers as a multimodal function. The 
two-dimension form is shown in Figure 3, and the formula of function evaluation in Equation 9. 


f(x) =10D+ 3 [x7 —10cos(2]] x,)] (9) 


i=] 


Rastrigin’s function has global minimum f(x )=0 at x =(0,0,...,0) with this constraint 


x,€[-5.12,5.12] for i=1,2, 3,...,.D 


Rastrigin functon 


f(x1,x2) 





Figure 3. Rastrigin Function of Two Dimensions 
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In this study, these control parameters presented in the Table 1 are used for both algorithms 
in the experiment. 


Table 1. Control Parameters 
Algorithms Parameters values 
=e W =0.7298; C, = C, = 2.05 


= N=25; P_ =0.25 


5. RESULTS 
Each of the two algorithms PSO and CS were tested on three test functions namely, Ackley, 


Rastrigin, and Rosenbrock for D=2,5,10,50,100 and 150. All implementation and analysis were performed in 
MATLAB. The two-algorithms had the same number of maximum iteration to perform a fair comparison [5]. 
The maximum iteration equaled to 3000.The comparison is based on solution accuracy, and runtime 
performance. 


5.1. Solution Accuracy 
It can be noticed from the presented results in figures 4,5, and 6 that the two algorithms PSO and CS 


had almost comparable results, i:e. f (x ) =() for D < 10 in all three functions. For the functions that have 


D> 50 CS achieved reasonable fitness value better than PSO. It can be concluded that CS, has more 
appropriate accuracy for problems containing dimensions up to 150 comparing with PSO. 
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Figure 4. Accuracy Comparing of CS and PSO on Figure 5. Accuracy Comparing of CS and PSO on 
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Figure 6. Accuracy Comparing of CS and PSO on Rastrigin function 
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5.2. Runtime Performance 

Analyzing the results in figure 7 and 8, it can be underattended that the runtimes of CS remain 
relatively low, while PSO runtime is increased within increases of dimensions. Consequently, optimization of 
the high dimensional problem has disturbed the performance of algorithm. 


particle swarm optimization cuckoo search optimization 
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Figure7. PSO Runtime Performance Figure 8. CS Runtime performance 


6. CONCLUSION 

In this paper, the authors have chosen tow algorithms based on swarm intelligence: 1-particle swarm 
optimization (PSO), 2-cuckoo search(CS) for comparison and analyzing on problems involving high 
dimensions. We have explained in details the functionality and pseudocode of the algorithms. Standard 
functions are using to evaluate the algorithms including Ackley, Rastrigin and Rosenbrock function. The 
algorithms have accomplished similar result for low dimensions problems. However, for higher dimensional 
problems up to 150, CS surpass PSO in terms of solution accuracy and runtime performances. 

To be noted that the comparison has been performed for an almost basic version of algorithms. The 
authors have a confidence that this paper will become a beneficial reference for researchers to work on the 
accuracy and run-time performance comparison of basic and modified algorithms for the optimization 
problems that contain high dimensions. 
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